{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tce3stUlHN0L"
      },
      "source": [
        "##### Copyright 2024 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "cellView": "form",
        "id": "tuOe1ymfHZPu"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yeadDkMiISin"
      },
      "source": [
        "# Gemini API: Model tuning with Python"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lEXQ3OwKIa-O"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://ai.google.dev/gemini-api/docs/model-tuning/python\"><img src=\"https://ai.google.dev/static/site-assets/images/docs/notebook-site-button.png\" height=\"32\" width=\"32\" />View on ai.google.dev</a>\n",
        "  </td>\n",
        "    <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/gemini-api/docs/model-tuning/python.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/google/generative-ai-docs/blob/main/site/en/gemini-api/docs/model-tuning/python.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jp_CKyzxUqx6"
      },
      "source": [
        "In this notebook, you'll learn how to get started with the tuning service using the Python client library for the Gemini API. Here, you'll learn how to tune the text model behind the Gemini API's text generation service."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sOz_wyZAlCuQ"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aHimx8NGMWDj"
      },
      "source": [
        "### Install the client library"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "cbcf72bcb56d"
      },
      "outputs": [],
      "source": [
        "!pip install -q google-generativeai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jdIYSl2kN0cq"
      },
      "source": [
        "### Import libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "8enrppafJPCX"
      },
      "outputs": [],
      "source": [
        "import google.generativeai as genai"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "lhqVUjH7ZKUi"
      },
      "outputs": [],
      "source": [
        "from google.colab import userdata\n",
        "genai.configure(api_key=userdata.get('GOOGLE_API_KEY'))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P-MYZECwlRCq"
      },
      "source": [
        "You can check you existing tuned models with the `genai.list_tuned_model` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "XyWzoYFxU4r6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tunedModels/number-generator-model-y330kq4bvrcv\n",
            "tunedModels/number-generator-model-d16t2al75c46\n",
            "tunedModels/number-generator-model-hte4uvo67uda\n",
            "tunedModels/number-generator-model-7ale2efhsgif\n",
            "tunedModels/number-generator-model-eoykb5s4p15s\n"
          ]
        }
      ],
      "source": [
        "for i, m in zip(range(5), genai.list_tuned_models()):\n",
        "  print(m.name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BhkXRzciv3Dp"
      },
      "source": [
        "## Create tuned model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OO8VZYAinLWc"
      },
      "source": [
        "To create a tuned model, you need to pass your dataset to the model in the `genai.create_tuned_model` method. You can do this be directly defining the input and output values in the call or importing from a file into a dataframe to pass to the method.\n",
        "\n",
        "For this example, you will tune a model to generate the next number in the sequence. For example, if the input is `1`, the model should output `2`. If the input is `one hundred`, the output should be `one hundred one`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "w-EBSe9wTbLB"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Model(name='models/gemini-1.5-flash-001-tuning',\n",
              "      base_model_id='',\n",
              "      version='001',\n",
              "      display_name='Gemini 1.5 Flash 001 Tuning',\n",
              "      description='Fast and versatile multimodal model for scaling across diverse tasks',\n",
              "      input_token_limit=16384,\n",
              "      output_token_limit=8192,\n",
              "      supported_generation_methods=['generateContent', 'countTokens', 'createTunedModel'],\n",
              "      temperature=1.0,\n",
              "      max_temperature=2.0,\n",
              "      top_p=0.95,\n",
              "      top_k=64)"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "base_model = [\n",
        "    m for m in genai.list_models()\n",
        "    if \"createTunedModel\" in m.supported_generation_methods and\n",
        "    \"flash\" in m.name][0]\n",
        "base_model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "baHjHh1oTTTC"
      },
      "outputs": [],
      "source": [
        "import random\n",
        "\n",
        "name = f'generate-num-{random.randint(0,10000)}'\n",
        "operation = genai.create_tuned_model(\n",
        "    # You can use a tuned model here too. Set `source_model=\"tunedModels/...\"`\n",
        "    source_model=base_model.name,\n",
        "    training_data=[\n",
        "        {\n",
        "             'text_input': '1',\n",
        "             'output': '2',\n",
        "        },{\n",
        "             'text_input': '3',\n",
        "             'output': '4',\n",
        "        },{\n",
        "             'text_input': '-3',\n",
        "             'output': '-2',\n",
        "        },{\n",
        "             'text_input': 'twenty two',\n",
        "             'output': 'twenty three',\n",
        "        },{\n",
        "             'text_input': 'two hundred',\n",
        "             'output': 'two hundred one',\n",
        "        },{\n",
        "             'text_input': 'ninety nine',\n",
        "             'output': 'one hundred',\n",
        "        },{\n",
        "             'text_input': '8',\n",
        "             'output': '9',\n",
        "        },{\n",
        "             'text_input': '-98',\n",
        "             'output': '-97',\n",
        "        },{\n",
        "             'text_input': '1,000',\n",
        "             'output': '1,001',\n",
        "        },{\n",
        "             'text_input': '10,100,000',\n",
        "             'output': '10,100,001',\n",
        "        },{\n",
        "             'text_input': 'thirteen',\n",
        "             'output': 'fourteen',\n",
        "        },{\n",
        "             'text_input': 'eighty',\n",
        "             'output': 'eighty one',\n",
        "        },{\n",
        "             'text_input': 'one',\n",
        "             'output': 'two',\n",
        "        },{\n",
        "             'text_input': 'three',\n",
        "             'output': 'four',\n",
        "        },{\n",
        "             'text_input': 'seven',\n",
        "             'output': 'eight',\n",
        "        }\n",
        "    ],\n",
        "    id = name,\n",
        "    epoch_count = 100,\n",
        "    batch_size=4,\n",
        "    learning_rate=0.001,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-As7ayWDK1w8"
      },
      "source": [
        "Your tuned model is immediately added to the list of tuned models, but its status is set to \"creating\" while the model is tuned."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "su64KgY4Uztj"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "TunedModel(name='tunedModels/generate-num-8122',\n",
              "           source_model='models/gemini-1.5-flash-001-tuning',\n",
              "           base_model='models/gemini-1.5-flash-001-tuning',\n",
              "           display_name='',\n",
              "           description='',\n",
              "           temperature=1.0,\n",
              "           top_p=0.95,\n",
              "           top_k=64,\n",
              "           state=<State.CREATING: 1>,\n",
              "           create_time=datetime.datetime(2024, 10, 21, 21, 42, 55, 748537, tzinfo=datetime.timezone.utc),\n",
              "           update_time=datetime.datetime(2024, 10, 21, 21, 42, 55, 748537, tzinfo=datetime.timezone.utc),\n",
              "           tuning_task=TuningTask(start_time=None,\n",
              "                                  complete_time=None,\n",
              "                                  snapshots=[],\n",
              "                                  hyperparameters=Hyperparameters(epoch_count=100,\n",
              "                                                                  batch_size=4,\n",
              "                                                                  learning_rate=0.001)),\n",
              "           reader_project_numbers=None)"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model = genai.get_tuned_model(f'tunedModels/{name}')\n",
        "\n",
        "model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "EUodUwZkKPi-"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<State.CREATING: 1>"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.state"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Pi8X5vkQv-3_"
      },
      "source": [
        "### Check tuning progress"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tWI-vAh4LJIz"
      },
      "source": [
        "Use `metadata` to check the state:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "g08vqtxYLMxT"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "total_steps: 375\n",
              "tuned_model: \"tunedModels/generate-num-8122\""
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "operation.metadata"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3lQ6gSMgK-kz"
      },
      "source": [
        "Wait for the training to finish using `operation.result()`, or `operation.wait_bar()`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "SOUowIv1HgSE"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "32da6e78695a43fe8dae58e9e6f55f76",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/375 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import time\n",
        "\n",
        "for status in operation.wait_bar():\n",
        "  time.sleep(30)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4cg868HzqOx5"
      },
      "source": [
        "You can cancel your tuning job any time using the `cancel()` method. Uncomment the line below and run the code cell to cancel your job before it finishes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "oQuJ70_hqJi9"
      },
      "outputs": [],
      "source": [
        "# operation.cancel()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lqiL0TWDqAPn"
      },
      "source": [
        "Once the tuning is complete, you can view the loss curve from the tuning results. The [loss curve](https://ai.google.dev/gemini-api/docs/model-tuning#recommended_configurations) shows how much the model's predictions deviate from the ideal outputs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "bIiG57xWLhP7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<Axes: xlabel='epoch', ylabel='mean_loss'>"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjMAAAGwCAYAAABcnuQpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABAvUlEQVR4nO3de5xbVb3H/e/aSSYz085Mb/QGUygUBemFS6FCQeFFERG5qRS0agWPHqRSSjkCBeEICIOcAw8iHFAfD8ij3ERBBSxCoSAcbi0XKZfS2tpW2mmh0850bplk7/X8sZPMpNch2ZmdTD9vX3m9OslOsrrB9stv/dZaxlprBQAAUKacsAcAAABQCMIMAAAoa4QZAABQ1ggzAACgrBFmAABAWSPMAACAskaYAQAAZS0a9gCKzfM8rV27VjU1NTLGhD0cAADQC9ZabdmyRaNHj5bj7Lz20u/DzNq1a1VfXx/2MAAAQB7WrFmjvfbaa6fX9PswU1NTI8m/GbW1tSGPBgAA9EZLS4vq6+uzf4/vTL8PM5mppdraWsIMAABlpjctIjQAAwCAskaYAQAAZY0wAwAAyhphBgAAlDXCDAAAKGuEGQAAUNYIMwAAoKwRZgAAQFkjzAAAgLJGmAEAAGWNMAMAAMoaYQYAAJQ1wgwAAChrhJkCuJ5V0vXCHgYAALs1wkwBNmzp1Jqm9rCHAQDAbo0wUwDXs3I9G/YwAADYrRFmCuQRZgAACBVhpkB0zAAAEC7CTIGspTIDAECYCDMFYpYJAIBwEWYK5FGZAQAgVISZAjHNBABAuAgzBWKaCQCAcBFmCkRhBgCAcBFmCkTPDAAA4SLMFMiKMAMAQJgIMwWyliZgAADCRJgpkBV9MwAAhIkwUyBrxUQTAAAhIswEgGkmAADCE2qYee6553TKKado9OjRMsbokUceyb6WTCZ16aWXasKECRowYIBGjx6tb37zm1q7dm14A94OKjMAAIQr1DDT1tamSZMm6fbbb9/mtfb2dr322mu68sor9dprr+kPf/iDli5dqlNPPTWEke6EpWcGAIAwRcP88pNOOkknnXTSdl+rq6vTk08+mfPcbbfdpiOOOEKrV6/WmDFjtvu+RCKhRCKR/bmlpSW4AW+HTf8PAACEo6x6Zpqbm2WM0aBBg3Z4TUNDg+rq6rKP+vr6oo6J1UwAAISrbMJMZ2enLr30Un31q19VbW3tDq+bN2+empubs481a9YUdVw0/wIAEK5Qp5l6K5lMavr06bLW6o477tjptfF4XPF4vI9G5ldmONIAAIDwlHyYyQSZVatW6emnn95pVSYMlgZgAABCVdJhJhNkli1bpmeeeUZDhw4Ne0jboP0XAIBwhRpmWltbtXz58uzPK1eu1BtvvKEhQ4Zo1KhR+spXvqLXXntNjz76qFzXVWNjoyRpyJAhqqioCGvYuTibCQCAUIUaZhYtWqTjjjsu+/PcuXMlSTNnztSPfvQj/elPf5IkHXzwwTnve+aZZ3Tsscf21TB3yopN8wAACFOoYebYY4/daVWjLCoe9MwAABCqslmaXaqsOM8AAIAwEWYK5E8zkWYAAAgLYaZA7AAMAEC4CDMF4tRsAADCRZgpFEuzAQAIFWEmAEQZAADCQ5gpEMcZAAAQLsJMgSwHGgAAECrCTIEsWwADABAqwkyByDIAAISLMFMwS88MAAAhIswUiB2AAQAIF2GmQKxmAgAgXISZglGXAQAgTISZAll2AAYAIFSEmQJx0CQAAOEizBSKIAMAQKgIMwWyLM0GACBUhJkCWUsLMAAAYSLMFMhK8jzCDAAAYSHMBIAsAwBAeAgzAWCaCQCA8BBmAsA0EwAA4SHMBMALewAAAOzGCDMB8EgzAACEhjATAHpmAAAID2EmAPTMAAAQHsJMAMgyAACEhzATAI4zAAAgPISZAHisZwIAIDSEmQCwmgkAgPAQZgLALBMAAOEhzASA1UwAAISHMBMAogwAAOEhzATAWsmypAkAgFAQZoJgWZ4NAEBYCDMBsBxoAABAaAgzAbBimgkAgLAQZgJgRRMwAABhIcwEwVp6ZgAACAlhJgB+ZYY0AwBAGEINM88995xOOeUUjR49WsYYPfLIIzmvW2t11VVXadSoUaqqqtK0adO0bNmycAa7E5bVTAAAhCbUMNPW1qZJkybp9ttv3+7rN954o2699VbdeeedevnllzVgwACdeOKJ6uzs7OOR7hxVGQAAwhMN88tPOukknXTSSdt9zVqrW265RT/84Q912mmnSZLuuecejRgxQo888ojOPvvsvhzqDhkZ9pkBACBEJdszs3LlSjU2NmratGnZ5+rq6jRlyhS9+OKLO3xfIpFQS0tLzqOYjOiZAQAgTCUbZhobGyVJI0aMyHl+xIgR2de2p6GhQXV1ddlHfX19UceZSTNUZgAACEfJhpl8zZs3T83NzdnHmjVrivp9RpJHXQYAgNCUbJgZOXKkJGn9+vU5z69fvz772vbE43HV1tbmPIrJGMMOwAAAhKhkw8zYsWM1cuRILViwIPtcS0uLXn75ZR155JEhjmxb1rIDMAAAYQl1NVNra6uWL1+e/XnlypV64403NGTIEI0ZM0Zz5szRj3/8Y+2///4aO3asrrzySo0ePVqnn356eIPeiqFnBgCAUIUaZhYtWqTjjjsu+/PcuXMlSTNnztTdd9+tSy65RG1tbfrud7+rzZs36+ijj9b8+fNVWVkZ1pC3YTK/IMwAABAKY/t5s0dLS4vq6urU3NwceP/Mvza1a8m/mhVxHB0+drAGVVcE+vkAAOyuPs7f3yXbM1M2jJEVB00CABAWwkyBjGgABgAgTISZIBiWZgMAEBbCTBAsm+YBABAWwkwQjKFnBgCAkBBmAkJtBgCAcBBm8mStVcr1lHQ9GdEBDABAWAgzebrj2X/o2P9+Vve+slry4wwAAAgBYSZPFRH/1qVcmz5oMtzxAACwuyLM5Kkimg4znieJnhkAAMJCmMlTpjKTdK3EDsAAAISGMJOnTGXG9azomQEAIDyEmTxlp5lcT7LsAAwAQFgIM3nKTjN5VjJMMwEAEBbCTJ5yKzPsAAwAQFgIM3nqXs1k/YMm6ZoBACAUhJk8xaPd+8z4PTMhDwgAgN0UYSZPFZGIJH+fGWMkjzQDAEAoCDN5quhRmTEyhBkAAEJCmMlTJswk05UZsgwAAOEgzOQptzLDNBMAAGEhzOQpFjGS/NVMVlYeWQYAgFAQZvIUTzcAS5LnsQMwAABhIczkKTPNJEkpa5lmAgAgJISZPPUMM67rMc0EAEBICDN5ijhGEZPum3E5mwkAgLAQZgoQi/phxmUHYAAAQkOYKUAs0n3YpCcv5NEAALB7IswUoCLSfdikR5YBACAUhJkCZCsznseZ2QAAhIQwU4BMz4xfmSHOAAAQBsJMAbLTTK6lMgMAQEgIMwXo2QBsLbsAAwAQBsJMAWI9GoDF8mwAAEJBmClARc+eGTHVBABAGAgzBYg6uauZmGYCAKDvEWYKkDmfKZXeApgoAwBA3yPMFGDrfWYozAAA0PcIMwWIRfyemWT6oElqMwAA9D3CTAFy95nh5GwAAMJAmClALNo9zURRBgCAcBBmChDbagdgKjMAAPS9kg4zruvqyiuv1NixY1VVVaX99ttP1157bcksga6IZPaZ8Ssz9MwAAND3omEPYGd+8pOf6I477tCvf/1rHXTQQVq0aJHOOecc1dXVafbs2WEPL1uZSdIzAwBAaEo6zPzf//2fTjvtNJ188smSpH322Uf33XefXnnllZBH5uvumUlPM4U7HAAAdkslPc101FFHacGCBXr//fclSW+++aaef/55nXTSSTt8TyKRUEtLS86jWCqylRl2AAYAICwlXZm57LLL1NLSogMOOECRSESu6+q6667TjBkzdviehoYGXX311X0yvm1Oze6TbwUAAD2VdGXmwQcf1G9/+1vde++9eu211/TrX/9a//3f/61f//rXO3zPvHnz1NzcnH2sWbOmaOOr6LFpHqdmAwAQjpKuzPzgBz/QZZddprPPPluSNGHCBK1atUoNDQ2aOXPmdt8Tj8cVj8f7ZHyZnpmkm04xhBkAAPpcSVdm2tvb5Ti5Q4xEIvI8L6QR5co5NZul2QAAhKKkKzOnnHKKrrvuOo0ZM0YHHXSQXn/9dd18880699xzwx6aJKkimplm8liaDQBASEo6zPzsZz/TlVdeqfPPP18bNmzQ6NGj9e///u+66qqrwh6apK12AKYBGACAUJR0mKmpqdEtt9yiW265JeyhbFfPpdkyLM0GACAMJd0zU+p67gAsS8cMAABhIMwUoHs1kycZI4/KDAAAfY4wU4DugyZZmg0AQFgCCTMtLS165JFH9O677wbxcWUj1qNnxrAwGwCAUOQVZqZPn67bbrtNktTR0aHJkydr+vTpmjhxon7/+98HOsBSljPNJMPSbAAAQpBXmHnuued0zDHHSJIefvhhWWu1efNm3Xrrrfrxj38c6ABLWUWPBmDPWmozAACEIK8w09zcrCFDhkiS5s+fry9/+cuqrq7WySefrGXLlgU6wFIWS/fMSJLnsWkeAABhyCvM1NfX68UXX1RbW5vmz5+vz33uc5KkTZs2qbKyMtABlrJMZUaSUizNBgAgFHltmjdnzhzNmDFDAwcO1N57761jjz1Wkj/9NGHChCDHV9IyPTOSlExZNs0DACAEeYWZ888/X0cccYTWrFmjE044IXsY5L777rtb9cw4xsgxkme7D5sEAAB9K+/jDCZPnqzJkydLklzX1VtvvaWjjjpKgwcPDmxw5SAWcZRIeUq5JBkAAMKQV8/MnDlz9Ktf/UqSH2Q++9nP6tBDD1V9fb0WLlwY5PhKXsRJn5xNZQYAgFDkFWYeeughTZo0SZL05z//WStXrtR7772niy66SFdccUWgAyx10XSYcV2WZgMAEIa8wsxHH32kkSNHSpIef/xxnXnmmfrEJz6hc889V2+99VagAyx1PQ+bpDIDAEDfyyvMjBgxQu+8845c19X8+fN1wgknSJLa29sViUQCHWCpy1ZmPMtBkwAAhCCvBuBzzjlH06dP16hRo2SM0bRp0yRJL7/8sg444IBAB1jqounKTMrzCDMAAIQgrzDzox/9SOPHj9eaNWt05plnKh6PS5IikYguu+yyQAdY6jKVmRQ7AAMAEIq8l2Z/5Stf2ea5mTNnFjSYchSNdDcAU5kBAKDv5dUzI0nPPvusTjnlFI0bN07jxo3Tqaeeqr/97W9Bjq0s5E4zhTwYAAB2Q3mFmd/85jeaNm2aqqurNXv2bM2ePVtVVVU6/vjjde+99wY9xpIWy04zieMMAAAIQV7TTNddd51uvPFGXXTRRdnnZs+erZtvvlnXXnutvva1rwU2wFJHAzAAAOHKqzKzYsUKnXLKKds8f+qpp2rlypUFD6qc9Nw0j2kmAAD6Xl5hpr6+XgsWLNjm+aeeekr19fUFD6qcZBqAOc4AAIBw5DXNdPHFF2v27Nl64403dNRRR0mSXnjhBd1999366U9/GugAS100fWJ4itVMAACEIq8w873vfU8jR47UTTfdpAcffFCSdOCBB+qBBx7QaaedFugAS12mMpNiB2AAAEKR9z4zZ5xxhs4444wgx1KWujfNY5oJAIAw5L3PDHxMMwEAEK5eV2YGDx4sY0yvrm1qasp7QOUmd5op5MEAALAb6nWYueWWW4o4jPIVS1dmki7TTAAAhKHXYSafc5duuOEGnXfeeRo0aNDHfm+5yFZmXCtZfxfg3lawAABA4YraM3P99df3+ymnnAZgcXI2AAB9rahhZnc4qyh7nIFrZSX1/98xAAClhdVMBequzKTDzG4Q4AAAKCWEmQLFspUZT7KWygwAAH2MMFOgTGUmma3MhDseAAB2N4SZAnWvZvKXZlObAQCgbxU1zBxzzDGqqqoq5leELprdZ8aymgkAgBDkfTaT53lavny5NmzYIM/zcl77zGc+I0l6/PHHCxtdGejeAdhjKRMAACHIK8y89NJL+trXvqZVq1Zts3rHGCPXdQMZXDmI5VRm6JkBAKCv5RVmzjvvPE2ePFmPPfaYRo0atVvveBvp0TMjemYAAOhzeYWZZcuW6aGHHtK4ceOCHk/Zya5momcGAIBQ5NUAPGXKFC1fvjzosZSlzD4zSdeTleSRZgAA6FN5VWYuuOACXXzxxWpsbNSECRMUi8VyXp84cWIgg5OkDz74QJdeeqn+8pe/qL29XePGjdNdd92lyZMnB/YdhYhFuncA9tg0DwCAPpdXmPnyl78sSTr33HOzzxljsidGB9UAvGnTJk2dOlXHHXec/vKXv2iPPfbQsmXLNHjw4EA+v1COMYpEun9OukwzAQDQ1/IKMytXrgx6HNv1k5/8RPX19brrrruyz40dO7ZPvrs3HGMUdbrTTCplWZ4NAEAfyyvM7L333kGPY7v+9Kc/6cQTT9SZZ56pZ599VnvuuafOP/98fec739nhexKJhBKJRPbnlpaWoo3PMVKsR2Um5XmsZgIAoI/lvWmeJL3zzjtavXq1urq6cp4/9dRTCxpUxooVK3THHXdo7ty5uvzyy/Xqq69q9uzZqqio0MyZM7f7noaGBl199dWBfP+uGGNkjKOIY+R6lmkmAABCkFeYWbFihc444wy99dZb2V4ZSdn9ZoLqmfE8T5MnT9b1118vSTrkkEO0ZMkS3XnnnTsMM/PmzdPcuXOzP7e0tKi+vj6Q8WwtvSpbsUgmzHjUZQAA6GN5Lc2+8MILNXbsWG3YsEHV1dV6++239dxzz2ny5MlauHBhYIMbNWqUPvWpT+U8d+CBB2r16tU7fE88HldtbW3Oo1gcY2TUvQtwyvO22REZAAAUV16VmRdffFFPP/20hg0bJsdx5DiOjj76aDU0NGj27Nl6/fXXAxnc1KlTtXTp0pzn3n///T7r2dkVxxi/CTi7CzAdMwAA9LW8KjOu66qmpkaSNGzYMK1du1aS3xi8dfgoxEUXXaSXXnpJ119/vZYvX657771Xv/jFLzRr1qzAvqMQxpFkem6cR88MAAB9La/KzPjx4/Xmm29q7NixmjJlim688UZVVFToF7/4hfbdd9/ABnf44Yfr4Ycf1rx583TNNddo7NixuuWWWzRjxozAvqMQfmWmO8ywmgkAgL6XV5j54Q9/qLa2NknSNddcoy9+8Ys65phjNHToUD3wwAOBDvCLX/yivvjFLwb6mUFxjOQod5qJLAMAQN/KK8yceOKJ2V+PGzdO7733npqamjR48ODd6gRtxxgZ033YpF+ZAQAAfSmvnpmM5cuX64knnlBHR4eGDBkS1JjKhjH+cvSoQ88MAABhySvMbNy4Uccff7w+8YlP6Atf+ILWrVsnSfr2t7+tiy++ONABlrJMz0y0x2GT9MwAANC38gozF110kWKxmFavXq3q6urs82eddZbmz58f2OBKnT/NZLL7zCRdj8oMAAB9LK+emb/+9a964okntNdee+U8v//++2vVqlWBDKwcbNMA7FGXAQCgr+VVmWlra8upyGQ0NTUpHo8XPKhyYYyRcXo0ALvsAAwAQF/LK8wcc8wxuueee7I/G2PkeZ5uvPFGHXfccYENrhxEHaNIpMdqJrIMAAB9Kq9pphtvvFHHH3+8Fi1apK6uLl1yySV6++231dTUpBdeeCHoMZa0iNO9mimVCnkwAADshvKqzIwfP15Lly7V0UcfrdNOO01tbW360pe+pNdff1377bdf0GMsaX6YSVdmLJUZAAD6Wl6VGUmqrKzUCSecoEmTJsnzPEnSq6++Kkk69dRTgxldGYj2DDMuS7MBAOhreYWZ+fPn6xvf+Iaampq2aXg1xsh13UAGVw4iTu5qJs8jzAAA0Jfymma64IILNH36dK1du1ae5+U8dqcgI/lhJpLpmXGtyDIAAPStvMLM+vXrNXfuXI0YMSLo8ZSdqONsdTYTaQYAgL6UV5j5yle+ooULFwY8lPLkmB5Ls12mmQAA6Gt59czcdtttOvPMM/W3v/1NEyZMUCwWy3l99uzZgQyuHBgjxXpUZryQxwMAwO4mrzBz33336a9//asqKyu1cOFCGWOyrxljdqsw4/TsmfGsPNIMAAB9Kq8wc8UVV+jqq6/WZZddJsfJa6aq38g5NZul2QAA9Lm8kkhXV5fOOuus3T7ISH7PTCySqcx49MwAANDH8kojM2fO1AMPPBD0WMqSMdpq0zwAANCX8ppmcl1XN954o5544glNnDhxmwbgm2++OZDBlQO/MuOHmaTLcQYAAPS1vMLMW2+9pUMOOUSStGTJkpzXejYD7w4c0+OgSc/KtXQAAwDQl/IKM88880zQ4yhbjpEqoj12ACbLAADQp+jgLZAxPc9m8uiZAQCgjxFmCuQYqSLSc58Z4gwAAH2JMFMgx5hsmEm6VGYAAOhrhJkC9VzNlHJt0VYzeZ5VU1tXcT4cAIAyRpgpkDFSRTQiyZ9msp6VLUKiae1K6V+b2tWVosMYAICeCDMFcoxRRbR7OXrS81SMthnrpVdLsZENAAA5CDMFcowUi0SyP3e5XlEqM57197BxaTAGACAHYaZAPRuAJSlZpCMNPGvlepJLZQYAgByEmQI5jlEkYhQxxT3SwLMs/QYAYHsIMwGION0b5/nLs4MPHFZWrlucfhwAAMoZYSYAEUfdYSZVnOXZ1vpTTPTMAACQizATgIhxFEsfNpn03KJ8h2etXFesZgIAYCuEmQD0RWXGs2I1EwAA20GYCUDEcRTNVmaK0zPjedZvAKYyAwBADsJMAKI9GoBTReyZ8VczBf/ZAACUM8JMACKOUdRJTzN5xTls0rPW3wWYNAMAQA7CTAD8pdmZk7OLczZTplcm6RJmAADoiTATAMcUvzLjpisyKRqAAQDIQZgJgDHFX83kpj8z5RJmAADoiTATAL8y49/KlOupGKWZTGWGaSYAAHKVVZi54YYbZIzRnDlzwh5KDscYxXocZ1CM5dOu5+9nwzQTAAC5yibMvPrqq/r5z3+uiRMnhj2UbTjGbwKW/DBTjMDhWk8Rx5HHYZMAAOQoizDT2tqqGTNm6Je//KUGDx6802sTiYRaWlpyHsVmejQAF2tjO9eTIsbIk39GEwAA8JVFmJk1a5ZOPvlkTZs2bZfXNjQ0qK6uLvuor68v+vgco5yl2cU4csD1rCIRI8/jsEkAAHoq+TBz//3367XXXlNDQ0Ovrp83b56am5uzjzVr1hR5hH7PTEUk0wAcfNiw6Q3zIsb4m+eRZQAAyIqGPYCdWbNmjS688EI9+eSTqqys7NV74vG44vF4kUeWyzE9jjPwgm8A9qxkZRV1jFLWY5oJAIAeSjrMLF68WBs2bNChhx6afc51XT333HO67bbblEgkFIlEQhyhz3GkWLoy43o28OXTnrXyJDmOkZcS00wAAPRQ0mHm+OOP11tvvZXz3DnnnKMDDjhAl156aUkEGSl3aXbKs0oGvLGdP7VkFXMcWU+sZgIAoIeSDjM1NTUaP358znMDBgzQ0KFDt3k+TH6YKV5lxlo/0DjGyFNxVksBAFCuSr4BuBwY0z3NlHJt4CdbW+s/jF/8oWcGAIAeSroysz0LFy4MewjbcIxRRTRdmbGekqniTDM5xshICjgrAQBQ1qjMBMDZqjLjef5y6qBkGoCN8Vc1UZkBAKAbYSYA/j4z3Q3AroLda8ZLTzM56XkmGoABAOhGmAmA3zPjr6xKuZ6/S2+A1RNrrZTumTEyNAADANADYSYAxhhVRNMHTabPZgqyr8XrsZrJGBP4aikAAMoZYSYglbEePTM22BVNnrXK1GIcqSincgMAUK4IMwGJR9PTTJ4nzwt2xVHPaSXHMUqmqMwAAJBBmAlIRY9Tsz0b8IojK5l0bcYxRswyAQDQjTATkMqYX5lJup6sLcJqJvk9OY6Rkmw0AwBAFmEmID17ZoxRoCuO/M/yP88Yo5RrA93HBgCAckaYCUhOZUYm4MqMlUlXZiLGX5pNDzAAAD7CTECqYpkGYOvvMxNg2vC3mUn3zDgKfBoLAIByRpgJSDwdZiT/IMjgp5kyPTMm8M8HAKCcEWYCktk0T5Jc1wa6sZ3rZdp/0+czecH25AAAUM4IMwHJ7DMj+WcnJd2Ae2ZMj8pMwGc/AQBQzggzAYk6jiJO+iBIKdDKTMqz2X9QjjF+ZYbV2QAASCLMBMYxUjQdZlwv4OMMvJ6VGaVXM1GZAQBAIswExhijaCRdmfGskqngwkbKs0rnJBlj/NVMhBkAACQRZgLjmO4jDbrczPlMwQQOz1O2MiP5TcBBfTYAAOWOMBMQxxgNiEclSW1drt+kG1D1JOV5MjnPGCozAACkEWYC4hijgekw096VCnTjPNfzj0jIsBI7AAMAkEaYCYhxlA0zbYlUoE26ns2dZpJlmgkAgAzCTEB6VmZa02EmiMqM5/mHSjo9sowxYp8ZAADSCDMBcYw0sDITZtx0A3Dhn+tZ/1SmnpUZx5hA97EBAKCcEWYC4hijmsw0U2fSr8wEMM3kWW23MhPkDsMAAJQzwkxATI/KzJZEKrCTrf3eG8n0WM8UMSbQTfkAAChnhJmAOMaoJh1mWjpT/l4wAVRmbLoy07P/1yHMAACQRZgJiGOMaitjkqQtnUlZmUAqM1ZW1io3zDgm0B2GAQAoZ4SZgDhGqkmHmdbOlGSDWXHkWcmTldNzB2D5zcWWjfMAACDMBMUYo9qqHjsAB7TPTHfPTDfHMf4Ow70ISynXY+UTAKBfI8wEKDPNJEkdCTeQEGE9SXbrpdnq9WGT67cktHpjW8HjAACgVBFmAhSPRlRdEZHkH2kQxPJpz/o9M85WDcD+Znq7fn8y5ak96RY8DgAAShVhJkBOjyMNOpLBVGYyxyJsvWleb3cY7kq56koxzQQA6L8IMwGKOkYDKrr7ZoJYPu3nldzQ4hjJterVNFOX67HBHgCgXyPMBCjiGA2Ip6eZEqlAlk/brddly28A7u1hk12uJzfAE7wBACg1hJkA+WGmuzLjn89UWIiwyl3JJPnTTK5n1ZuP7kr5QYZN9gAA/RVhJkBRx+kRZlL+8ukCl2dnDprcRi9OznY9K8+zgZ3gDQBAKSLMBMgxJruaqbUzJS+A6Z0dvd1o18clpDxPbnoJN2EGANBfEWYCZEz3aqbWRCq7EqkQnme3mWbK2FVA8Tz/GusFsxsxAACliDATIMcx2ZOz29JhptAQYe3Wa5m6n+9NZcazVikagAEA/RhhJkCOUXZpdmsilW4ALuwzvR2lGbPrz/Z7ZvzDKgkzAID+ijATIMd0L83e0pmuzBQ4zZTyvJxDJrPfJbPLFUopz/9+xxilCDMAgH6KMBMgY6SB6crMls6Uf35SoQ3Ant16m5nsd+1qh2G/38ZKVGYAAP1YyYeZhoYGHX744aqpqdHw4cN1+umna+nSpWEPa7v8yowfZrpcT12uV3ADcMrbZs+87HftqtqS8qysjIwMYQYA0G+VfJh59tlnNWvWLL300kt68sknlUwm9bnPfU5tbaV3ErRjjCpjjiLpUyHbEm4AS7N3MM1kpNQujinwv9ufZupKcdgkAKB/ioY9gF2ZP39+zs933323hg8frsWLF+szn/nMNtcnEgklEonszy0tLUUfY4ZjpIjjaGA8quaOpFoTqYLDjOttuwOw5K+c2tU0U8r1ZGQUcYy6Ajj0EgCAUlTylZmtNTc3S5KGDBmy3dcbGhpUV1eXfdTX1/fZ2IwxkpFqtlqeXQh3h5WZXU8zdbmeIsb4lRnCDACgnyqrMON5nubMmaOpU6dq/Pjx271m3rx5am5uzj7WrFnTZ+NzjP+oSffNdHS5u6ye7IrrabulGcf4Db47O/spkfLkOOnKTACHXgIAUIpKfpqpp1mzZmnJkiV6/vnnd3hNPB5XPB7vw1F1c4yRI6OBlTFJ/mGTyV30teyK69mdVGb84wqcHewRnExXZiLGZI9WyPTzAADQX5RNZeb73/++Hn30UT3zzDPaa6+9wh7OdjnGpI808Peaae9KFVSZsdY/imC7PTPGyLM7P6Yg6Vo5jpHjiPOZAAD9VslXZqy1uuCCC/Twww9r4cKFGjt2bNhD2iFj/L6ZzPlMbQl3lxvb7YxnJU/br8wYkz5Rewf5JOV6fiUm3TPjcqQBAKCfKvkwM2vWLN1777364x//qJqaGjU2NkqS6urqVFVVFfLocjnGyDHqPp+pK6VkAb0qnrWy2v4+MxHH7HSH4UwlJho12Wv9YBXJezwAAJSikp9muuOOO9Tc3Kxjjz1Wo0aNyj4eeOCBsIe2DSddmcmcz9SWPZ8pv0DjV162vwOwY0z2VOztcT3/xO6IY7LXFnpOFAAApajkKzO2wKXNfSmSXjlU3eOwSVd2p026O5M5GXtHm+ZZu+Og1PNcptzKDAAA/UvJV2bKiTFGldGIqmL+bW3tTGVXEeUj0xOz/bOZjDzZHe5j43lWnqvs6iUrFXzoJQAApYgwE7CqCkfVFX5fSmt607x8N87zrF+ZMjup6uwooPjnMuVWdWgABgD0R4SZgFXGIqrsEWZcz8s7RFhr5cmv0Pz4sXf0//5tRc7rRjvug8mcy9T9Ybs+ywkAgHJEmAlYLOJoQHqfGc/6y7PzbVXxrGQ9aV1zp15e2aQ/vblWncnuAyOtdrKaybM9o0x2KTcAAP0NYSZg0YhRLBJRPOrf2pbOVN69KjbdM7OxzT8400pa3dSec82OGoBdz0q2e4rJPzmbBmAAQP9DmAlYzHHkyHQfNtmZLKAB2K++NLV1ZZ/758a27K+NdnxydtL1chqHOWwSANBfEWYCFos6ikakmuz5TPmfnJ1538bW7jCzamN3ZSYWcdSaSG33vYmUfy5TRsQxBW3gBwBAqSLMBCzqGEUcRwOyTcBuYUuzJX3Umsg+17MyE4s4au9yt7sXT9L1cg6VdIyUSLnbXAcAQLkjzASsIuIo6hgN6HE+U/6rmSQjq492WJkxSrnedk/m7kp5WtvcoUt+/3e9tnqTIo5/PlO+uxEDAFCqCDMBcxyjiqjTHWYKmGayVrIy2tijMtPckdSmdj/cRB1HSdfbphfGWquka/XSiia9u65F85c09tgFmDADAOhfCDNFUBlzVBXzp5naEu4Om3R3xQ9BVh+lVzPFIv60UaY6E4sYJV27zSolN32UwYfpELSuuSN7cjbLswEA/Q1hpgiqYtHsLsDtidR2p4F6w7NWnV2e2hJ+r8v40XWSpH9+5PfNmHSD79ZhJpWeTtrQ0ilJWtvcmd1nhsoMAKC/IcwUQUXUyZ6c3dqVyrsyY60/rSRJ1RURHTiqVlJuE7CkbT7fs36YWd/iV2a6Up6a25NyPcllF2AAQD9DmCmCWMTRwMp0ZabLVSLPzeo8a7P9MUMHxrX30GpJuU3AjjFq78pdnp3yrFoSyZxl240tndnpJwAA+hPCTBHEIiZbmWlLpJRIunntvut6Vpva/crMsAEV2mfoAEn+LsCZFVKxiKO2rcKM61ptaE7kPLeuuVMyUirfsxUAAChRhJkiiEYc1VT5m+a1JlLqcr289nhxPavN6crMsIFxjaitVEXUUZfrqbHZ74eJRYw6u3IPs0x5Vhu2dOZ81rrmDkk7PpgSAIByRZgpgoqIo7oqvzLT3uWqs8tVZzKPyoy12pyuzAwdWKGIYzRmiD/VlOmbiUX85dk9+2Y82703TWbbvLWbO/2Ts0kzAIB+hjBTBNGIUV1VRfbnti4378rMph6VGUnaJ9s344eZqGOU9GxOX07K616Wvd/wgZLSlRmjvDfwAwCgVBFmiiCa3jgvszy7o8tVR1e+Yaa7MiNJe6f7Zv6ZbgKORhy5rpfTk+O6Vh9u8cPMYWMGS0ovz9a2K58AACh3hJkiMMaoKhbRwPQuwImUp5bO5Mf+HM+TNqVPzB42IFOZyYSZHsuzjXJ2AU66XjbMTNyrTo7xl2e3dCTzakQGAKCUEWaKpDLWfaRBZ9JVR9L72FWR9q6UtqSXV289zdTY3KnOpF/tMTLqSro572tKT0/tOahKI2orJfkHVua7gR8AAKWKMFMklbFI9uTszqSrZOrj7zeT6Xvxz3ryP2tQdYUGVcVk5S/Rlvwzmnouz/5gU4es9d83ZECFRtVV+Z+3JbHNOU4AAJQ7wkyRxCLdlZn2LldJ1yqR7H3fjO2xImnYgIrs0QWSemyel1nRZLJHHkjSvzb7y7BH1lbKGKPRdX5lZsOWhFIpTs4GAPQvhJkiiUZMNsxs6UzJSh+rMmOtsqdlZ6aYMrZuAs4sz+5KebLWam2PMCNJowb5lZn1LQm51mMXYABAv0KYKZKKiJNtAN6SSMlI2xw7sDOetdrYljnKoCLnta2XZ/fcayblWTWmD5gcma7IZCoz61s65VrL8mwAQL9CmCmSaMRRbWWmMpNURcRRa2fvp5k8KzW17bwysyq7PNso5forllzPZncHHpUJM+nKTGNLp1zXI8wAAPoVwkyRRB2jQdV+RWV9S6cqoo7au1K9DhKetWpqzewxkxtmxgyplpG0uSOpTe1dcoyRlaeudFDJLMvOTDMNr4nLMf40V1N7UinCDACgHyHMFElFxNGnRtVKklZ82Kb2LvdjndFkrdTUnqnM5E4zVcYi2SmkTHXGyq/MJF1PGzJhJn1NNOJkl2eva+6kMgMA6FcIM0XiOEbDa+MaM6RaVtLba5vVlfJ6fUaTZ62a2tKVmQHxbV7Pbp73kd83EzGOOrpS2tjapUTKk5GyAUbqnnLa0JIgzAAA+hXCTBFVxhwdNNqvzry+ZnN6RVPvKjOJpKuWDj/MbF2ZkaRPjqyRJC1evUmSP63V3uVldwYeOjCuWKT7H+/o7F4zVGYAAP0LYaaIqiui2amm11dvlrVWnb08o2lDa0JWfkiprYpt8/qR+w6VJP39X5vV3JFULOKoo8vNTjtlKjEZowZ17zXD0mwAQH9CmCmiWMTRvsMGqCoWUXNHUus2d2aPJ9iVxs3+iqShAyvk9NgwL2P0oCrtO2yAPCu9tGKjYhFHXZ6rVU1+ZWZkbW6YyVRmNmxJyOVIAwBAP0KYKaKKqKNY1NGk+jpJ0jvrWtTW6fZqB97MXjHb65fJmDpumCTp//7xkWIRo6RrtaYpvWHe1pWZTJhpSSiRyg1UWzqT2eXcAACUG8JMEUUdv6JyaP1gSdLfP2hWl9u7M5oyYWbrPWZ6mrqfH2be/FezOpKuXNfTumY/zGw9zTS81l+e3eV6amxOZJ/3PKsVH7Zp2YYt6ujlFBgAAKWEMFNE0YgjxzGatJdfmVm2fos2tyd71QS8Phtmtm3+zdhzcJX2GVot17N6eWWTZEy2wjJiq2mmWMTR8Br/uTWb2rPPf9ia0PqWTrV2pvRRa0IAAJQbwkwRVUQcxRyjIQPj2mtwlTwrvbOuuVeVmfUtfrDYesO8rR2Vrs68sPwjdaU8bWr3V0BtXZmRpNGDMmHGr950pTyt2timivShmB9s7lCSU7UBAGWGMFNE0YhRJOIo5VkdNsafalryQYs6e3F69oZeVGYk6eh038wbazZnD5isroiopnLbFVCZvpkPNnXIWqvG5g41tXVpUHWFaitj2tzepY3pk7oBACgXhJkiikUcRR0j17M6dG8/zLyzrkUtHbsODJldfHfWMyNJ9UOqVT+kWinP6ql310vyjy/YnkxlprHFX1W1uqldqza265pH39az729QRSSidc0dvWpQBgCgVBBmiqwqFlHK9TR+dJ0qoo42tSf1bmOr7E72ekm5XrZCMnTAziszkjR1P3/PmUX/9DfQ23pZdkamMtPY3KF3PmjW7c8sV8Nf3tNrqzfrlqeWaen6Fm1s7dLm9GZ9AACUA8JMkVXG/GmmiqijCXv6jcCLV23aad/MR61dcq2VY5Q9rHJnMlNNmXg0Kn1K9tYye82s3dyp79yzWC+uaJKRtO8eA2Ql/T9PLdOKj1rVmF4RBQBAOSDMFFlVLJrdcTfTN/Pmms1K7OSMpszy6rqqmCLOthvmbW3MkGrt2SPA7LmDMJNZnp3yrLYkUhozpFo3fnmibj7zYB2292B1pTzd/sw/9PYHLdrSSXUGAFAeyiLM3H777dpnn31UWVmpKVOm6JVXXgl7SL0WjRhZK7me1WHpvpllG1r15r827TAwZJZXD+7FFJMkGWOyG+hJ226YlxFxjA4YWauoY3T24fW65ayDdcCoWkUco0tO/KT2HTZAzR1J3fTk+/rHhlZJ/j40LZ1J/WtTu/6xoVXN7cmdTpEBANDXomEPYFceeOABzZ07V3feeaemTJmiW265RSeeeKKWLl2q4cOHhz28XaqtimnowAo1tnSqtjKqUXWVWtfcqe/+f4t17Cf20Hc+s68OHTNYpseRBesyYaYXU0wZU/cbqgcXrZHU3TPjelab27vUmfJkjOTIaO60/eXJP9X7w9aEIsaoMhZRVSyiq774Kf3HQ2+qsaVTs+9/Q3sPqdaGLQk1tXdpS2dSdZUxjd1jgCbuNUhH7TdUU8YOUVVFyf8rBADo54wt8f/MnjJlig4//HDddtttkiTP81RfX68LLrhAl1122S7f39LSorq6OjU3N6u2trbYw92uRMrV2k0dWtXUrrfXNuu3L63W2nRgMcY/NPL4A4errqpCA+NR/fnva/XY39fphANHaPbx+/fqO6y1uvO5FXJdT987dj+1dKTUnnQ1ZEBMo+qqFI86/vEKEUfRiFFn0lNbIqWWjqQ2tSXVkUopmfK0trlTP5n/njp3Mg3WUyxiFI9GVBlzFI9GVFsV1ai6Ku05qEr1Q6o0orZSVbGIYlFH8Yg/Bmc7U2eOMYo6RpH0w7/EZO9R90/blwmDO78m8zm5V1n17v8CW7+v52eiNG39z6e3f9oF8c+1mN/Vl78PoDdq4jHVVW+7JUghPs7f3yUdZrq6ulRdXa2HHnpIp59+evb5mTNnavPmzfrjH/+4zXsSiYQSie6dbFtaWlRfXx9qmMlobk/qnxvbtK65Q2+vbdHT727Q2+tadnj9GQfvqbMOr5e04z+UrO1u/JWVkq6n9mRKg6orNGZItfaoiSsW2flsoutZtXWl1J5w1dzRpUWrNunVlU0aUBFVTWVMtVVRDYhH9VFrQqs2tmnFh23658Z2tfby0EwAQP/275/ZV/O+cGCgn/lxwkxJzxF89NFHcl1XI0aMyHl+xIgReu+997b7noaGBl199dV9MbyPra46pvGVdRpZV6kDR9XqCxNGafn6Vv3l7UZ91JpQR9JVZ5erzqSriqijz3ximFJeLyokxq8ZGEkVMUf7Dq/T8JpKVUR71xIVcYxqK2OqrYxpZF2lxg2v0amTRsvz/KpFJjB51irlWiVdT10pVxtaEupIppR0rZKeVVfSVXNnSo3NnWps6dTG1oQ2tSeVdL3s+1Kelbed/OxZvz/Hs1auZ7P/5Zm5cnvVk54fY3pcu6Nrcj/x4wkq8W/vPx2suitKOwutgXy/dl69CvJ9H2fIu/rskv0vLuxYvv+y9eX3F3OMpfbZQd2P9DVbF9ijkXDLgCUdZvIxb948zZ07N/tzpjJTKiKOyTk36cBRtfripFH+X+A7eM/O/iIzpudfhP70jCmwthxxzHZ3EN7afsNrev2Z1m4bULZ+PfOaH55sIH+BF6Pu2NtpqVJlbWlNP/TleHr7Xb25rhRq2uUwRvQ/sXTLQCkp6TAzbNgwRSIRrV+/Puf59evXa+TIkdt9TzweVzy+811zS40xJvRUW2zGmF38wdu/f/8AgOIprWi1lYqKCh122GFasGBB9jnP87RgwQIdeeSRIY4MAACUipKuzEjS3LlzNXPmTE2ePFlHHHGEbrnlFrW1temcc84Je2gAAKAElHyYOeuss/Thhx/qqquuUmNjow4++GDNnz9/m6ZgAACweyrppdlBKIV9ZgAAwMfzcf7+LumeGQAAgF0hzAAAgLJGmAEAAGWNMAMAAMoaYQYAAJQ1wgwAAChrhBkAAFDWCDMAAKCsEWYAAEBZK/njDAqV2eC4paUl5JEAAIDeyvy93ZuDCvp9mNmyZYskqb6+PuSRAACAj2vLli2qq6vb6TX9/mwmz/O0du1a1dTUyBiT9+e0tLSovr5ea9as4YynIuNe9x3udd/hXvcd7nXfKea9ttZqy5YtGj16tBxn510x/b4y4ziO9tprr8A+r7a2lv9z9BHudd/hXvcd7nXf4V73nWLd611VZDJoAAYAAGWNMAMAAMoaYaaX4vG4/vM//1PxeDzsofR73Ou+w73uO9zrvsO97julcq/7fQMwAADo36jMAACAskaYAQAAZY0wAwAAyhphBgAAlDXCTC/dfvvt2meffVRZWakpU6bolVdeCXtIZa2hoUGHH364ampqNHz4cJ1++ulaunRpzjWdnZ2aNWuWhg4dqoEDB+rLX/6y1q9fH9KI+48bbrhBxhjNmTMn+xz3OjgffPCBvv71r2vo0KGqqqrShAkTtGjRouzr1lpdddVVGjVqlKqqqjRt2jQtW7YsxBGXL9d1deWVV2rs2LGqqqrSfvvtp2uvvTbnLB/ud36ee+45nXLKKRo9erSMMXrkkUdyXu/NfW1qatKMGTNUW1urQYMG6dvf/rZaW1uLM2CLXbr//vttRUWF/d///V/79ttv2+985zt20KBBdv369WEPrWydeOKJ9q677rJLliyxb7zxhv3CF75gx4wZY1tbW7PXnHfeeba+vt4uWLDALlq0yH7605+2Rx11VIijLn+vvPKK3WeffezEiRPthRdemH2eex2MpqYmu/fee9tvfetb9uWXX7YrVqywTzzxhF2+fHn2mhtuuMHW1dXZRx55xL755pv21FNPtWPHjrUdHR0hjrw8XXfddXbo0KH20UcftStXrrS/+93v7MCBA+1Pf/rT7DXc7/w8/vjj9oorrrB/+MMfrCT78MMP57zem/v6+c9/3k6aNMm+9NJL9m9/+5sdN26c/epXv1qU8RJmeuGII46ws2bNyv7suq4dPXq0bWhoCHFU/cuGDRusJPvss89aa63dvHmzjcVi9ne/+132mnfffddKsi+++GJYwyxrW7Zssfvvv7998skn7Wc/+9lsmOFeB+fSSy+1Rx999A5f9zzPjhw50v7Xf/1X9rnNmzfbeDxu77vvvr4YYr9y8skn23PPPTfnuS996Ut2xowZ1lrud1C2DjO9ua/vvPOOlWRfffXV7DV/+ctfrDHGfvDBB4GPkWmmXejq6tLixYs1bdq07HOO42jatGl68cUXQxxZ/9Lc3CxJGjJkiCRp8eLFSiaTOff9gAMO0JgxY7jveZo1a5ZOPvnknHsqca+D9Kc//UmTJ0/WmWeeqeHDh+uQQw7RL3/5y+zrK1euVGNjY869rqur05QpU7jXeTjqqKO0YMECvf/++5KkN998U88//7xOOukkSdzvYunNfX3xxRc1aNAgTZ48OXvNtGnT5DiOXn755cDH1O8PmizURx99JNd1NWLEiJznR4wYoffeey+kUfUvnudpzpw5mjp1qsaPHy9JamxsVEVFhQYNGpRz7YgRI9TY2BjCKMvb/fffr9dee02vvvrqNq9xr4OzYsUK3XHHHZo7d64uv/xyvfrqq5o9e7YqKio0c+bM7P3c3p8n3OuP77LLLlNLS4sOOOAARSIRua6r6667TjNmzJAk7neR9Oa+NjY2avjw4TmvR6NRDRkypCj3njCD0M2aNUtLlizR888/H/ZQ+qU1a9bowgsv1JNPPqnKysqwh9OveZ6nyZMn6/rrr5ckHXLIIVqyZInuvPNOzZw5M+TR9T8PPvigfvvb3+ree+/VQQcdpDfeeENz5szR6NGjud+7GaaZdmHYsGGKRCLbrOxYv369Ro4cGdKo+o/vf//7evTRR/XMM89or732yj4/cuRIdXV1afPmzTnXc98/vsWLF2vDhg069NBDFY1GFY1G9eyzz+rWW29VNBrViBEjuNcBGTVqlD71qU/lPHfggQdq9erVkpS9n/x5Eowf/OAHuuyyy3T22WdrwoQJ+sY3vqGLLrpIDQ0NkrjfxdKb+zpy5Eht2LAh5/VUKqWmpqai3HvCzC5UVFTosMMO04IFC7LPeZ6nBQsW6MgjjwxxZOXNWqvvf//7evjhh/X0009r7NixOa8fdthhisViOfd96dKlWr16Nff9Yzr++OP11ltv6Y033sg+Jk+erBkzZmR/zb0OxtSpU7fZYuD999/X3nvvLUkaO3asRo4cmXOvW1pa9PLLL3Ov89De3i7Hyf1rLBKJyPM8SdzvYunNfT3yyCO1efNmLV68OHvN008/Lc/zNGXKlOAHFXhLcT90//3323g8bu+++277zjvv2O9+97t20KBBtrGxMeyhla3vfe97tq6uzi5cuNCuW7cu+2hvb89ec95559kxY8bYp59+2i5atMgeeeSR9sgjjwxx1P1Hz9VM1nKvg/LKK6/YaDRqr7vuOrts2TL729/+1lZXV9vf/OY32WtuuOEGO2jQIPvHP/7R/v3vf7ennXYaS4XzNHPmTLvnnntml2b/4Q9/sMOGDbOXXHJJ9hrud362bNliX3/9dfv6669bSfbmm2+2r7/+ul21apW1tnf39fOf/7w95JBD7Msvv2yff/55u//++7M0O2w/+9nP7JgxY2xFRYU94ogj7EsvvRT2kMqapO0+7rrrruw1HR0d9vzzz7eDBw+21dXV9owzzrDr1q0Lb9D9yNZhhnsdnD//+c92/PjxNh6P2wMOOMD+4he/yHnd8zx75ZVX2hEjRth4PG6PP/54u3Tp0pBGW95aWlrshRdeaMeMGWMrKyvtvvvua6+44gqbSCSy13C/8/PMM89s98/omTNnWmt7d183btxov/rVr9qBAwfa2tpae84559gtW7YUZbzG2h5bJQIAAJQZemYAAEBZI8wAAICyRpgBAABljTADAADKGmEGAACUNcIMAAAoa4QZAABQ1ggzAACgrBFmAOx2Fi5cKGPMNodrAihPhBkAAFDWCDMAAKCsEWYA9DnP89TQ0KCxY8eqqqpKkyZN0kMPPSSpewroscce08SJE1VZWalPf/rTWrJkSc5n/P73v9dBBx2keDyuffbZRzfddFPO64lEQpdeeqnq6+sVj8c1btw4/epXv8q5ZvHixZo8ebKqq6t11FFHaenSpcX9jQMoCsIMgD7X0NCge+65R3feeafefvttXXTRRfr617+uZ599NnvND37wA91000169dVXtccee+iUU05RMpmU5IeQ6dOn6+yzz9Zbb72lH/3oR7ryyit19913Z9//zW9+U/fdd59uvfVWvfvuu/r5z3+ugQMH5ozjiiuu0E033aRFixYpGo3q3HPP7ZPfP4BgcWo2gD6VSCQ0ZMgQPfXUUzryyCOzz//bv/2b2tvb9d3vflfHHXec7r//fp111lmSpKamJu211166++67NX36dM2YMUMffvih/vrXv2bff8kll+ixxx7T22+/rffff1+f/OQn9eSTT2ratGnbjGHhwoU67rjj9NRTT+n444+XJD3++OM6+eST1dHRocrKyiLfBQBBojIDoE8tX75c7e3tOuGEEzRw4MDs45577tE//vGP7HU9g86QIUP0yU9+Uu+++64k6d1339XUqVNzPnfq1KlatmyZXNfVG2+8oUgkos9+9rM7HcvEiROzvx41apQkacOGDQX/HgH0rWjYAwCwe2ltbZUkPfbYY9pzzz1zXovH4zmBJl9VVVW9ui4Wi2V/bYyR5PfzACgvVGYA9KlPfepTisfjWr16tcaNG5fzqK+vz1730ksvZX+9adMmvf/++zrwwAMlSQceeKBeeOGFnM994YUX9IlPfEKRSEQTJkyQ53k5PTgA+i8qMwD6VE1Njf7jP/5DF110kTzP09FHH63m5ma98MILqq2t1d577y1JuuaaazR06FCNGDFCV1xxhYYNG6bTTz9dknTxxRfr8MMP17XXXquzzjpLL774om677Tb9z//8jyRpn3320cyZM3Xuuefq1ltv1aRJk7Rq1Spt2LBB06dPD+u3DqBICDMA+ty1116rPfbYQw0NDVqxYoUGDRqkQw89VJdffnl2mueGG27QhRdeqGXLlunggw/Wn//8Z1VUVEiSDj30UD344IO66qqrdO2112rUqFG65ppr9K1vfSv7HXfccYcuv/xynX/++dq4caPGjBmjyy+/PIzfLoAiYzUTgJKSWWm0adMmDRo0KOzhACgD9MwAAICyRpgBAABljWkmAABQ1qjMAACAskaYAQAAZY0wAwAAyhphBgAAlDXCDAAAKGuEGQAAUNYIMwAAoKwRZgAAQFn7/wEcUU570yTqDwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import pandas as pd\n",
        "import seaborn as sns\n",
        "\n",
        "model = operation.result()\n",
        "\n",
        "snapshots = pd.DataFrame(model.tuning_task.snapshots)\n",
        "\n",
        "sns.lineplot(data=snapshots, x = 'epoch', y='mean_loss')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rkoQTXb1vSBC"
      },
      "source": [
        "## Evaluate your model\n",
        "\n",
        "You can use the `genai.generate_content` method and specify the name of your model to test your model performance."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "zO0YcuSyxydZ"
      },
      "outputs": [],
      "source": [
        "model = genai.GenerativeModel(model_name=f'tunedModels/{name}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "UwGrrj6hS_x2"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'56'"
            ]
          },
          "execution_count": 15,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "result = model.generate_content('55')\n",
        "result.text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "YSNB2zjTx5SZ"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'123456'"
            ]
          },
          "execution_count": 16,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "result = model.generate_content('123455')\n",
        "result.text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "Y2YVO-m0Ut9H"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'five'"
            ]
          },
          "execution_count": 17,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "result = model.generate_content('four')\n",
        "result.text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "h2MkTR0uTb6U"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'cinq'"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "result = model.generate_content('quatre') # French 4\n",
        "result.text                               # French 5 is \"cinq\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "OruCW1zETsZw"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'IV'"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "result = model.generate_content('III')    # Roman numeral 3\n",
        "result.text                               # Roman numeral 4 is IV"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "thDdSuUDUJOx"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'八'"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "result = model.generate_content('七')  # Japanese 7\n",
        "result.text                            # Japanese 8 is 八!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HpIA1IFevQQR"
      },
      "source": [
        "It really seems to have picked up the task despite the limited examples, but \"next\" is a simple concept, see the [tuning guide](https://ai.google.dev/gemini-api/docs/model-tuning) for more guidance on improving performance."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nmuQCbTYwIOx"
      },
      "source": [
        "## Update the description\n",
        "\n",
        "You can update the description of your tuned model any time using the `genai.update_tuned_model` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "9gAVuXT_wG3x"
      },
      "outputs": [],
      "source": [
        "genai.update_tuned_model(f'tunedModels/{name}', {\"description\":\"This is my model.\"});"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "d-c3YerBxVYs"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'This is my model.'"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model = genai.get_tuned_model(f'tunedModels/{name}')\n",
        "\n",
        "model.description"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i_TpwvBB4bQ7"
      },
      "source": [
        "## Delete the model\n",
        "\n",
        "You can clean up your tuned model list by deleting models you no longer need. Use the `genai.delete_tuned_model` method to delete a model. If you canceled any tuning jobs, you may want to delete those as their performance may be unpredictable."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "cepfaUCvVGCo"
      },
      "outputs": [],
      "source": [
        "genai.delete_tuned_model(f'tunedModels/{name}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ljEssIshYDEr"
      },
      "source": [
        "The model no longer exists:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "kN_bkut_4ayL"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "<class 'google.api_core.exceptions.NotFound'>: 404 GET https://generativelanguage.googleapis.com/v1beta/tunedModels/generate-num-8122?%24alt=json%3Benum-encoding%3Dint: Tuned model tunedModels/generate-num-8122 does not exist.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "WARNING:tornado.access:404 GET /v1beta/tunedModels/generate-num-8122?%24alt=json%3Benum-encoding%3Dint (127.0.0.1) 838.85ms\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "  m = genai.get_tuned_model(f'tunedModels/{name}')\n",
        "  print(m)\n",
        "except Exception as e:\n",
        "  print(f\"{type(e)}: {e}\")"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "python.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
